1. Field of the Invention
The present invention relates generally to the field of computer software, and more particularly to a system and method for efficiently performing memory intensive computations including a bidirectional synchronization mechanism for maintaining consistency of data.
2. Background of the Invention
The intensive computations required in certain application software necessitate access to and manipulation of a substantial amount of data simultaneously. If all of the data is not resident in memory, computation speed will be significantly slowed down. In addition, the required data is typically closely tied to transactional data stored in a database. The data in the database might be changing frequently due to, for example, real world transactions. The disparity between the two data sets causes inaccuracies in the results of the computations.
These problems are encountered in many business applications. For example, supply chain planning exhibits such problems. Supply chain planning requires a large amount of information to build an enterprise plan. This information typically includes data related to production capacity, inventory balance, distribution network, bill of material (BOM), sourcing of components, etc. In order to build a good enterprise plan, a computation intensive mathematical algorithm should have instant access to this data in order to be able to run efficiently.
This information (such as inventory balance, etc.) is typically stored in a database. As mentioned herein, problems arise when live business transactions are performed against the database, thus causing data in the database to change at the same time as a mathematical algorithm is being run on the data. At least two prior art attempts were made with respect to trying to overcome the problem of the continuous change of data in the database.
Referring to FIG. 1, a first approach was to extract the required transactional data from a database 100 and store the data in a flat file 102. Computation software 104 would then read file 102 and process the data through the appropriate mathematical algorithm. The output from this computation would be written to another file 106 and exported to, or merged with, database 100.
Referring to FIG. 2, a second approach was to extract and transfer a snapshot of the required transactional data from a database 200 to a dedicated database 202. Computation software 204 would read the data directly from dedicated database 202 and process the data through the appropriate mathematical algorithm. The output from this computation would be written to dedicated database 202 and transferred to, or merged with, database 200. In this scenario, all business transactions are processed through database 200 but not through dedicated database 202. Dedicated database 202 acts as a medium to mitigate the constant changing nature of the data in database 200 while providing a stable view of the data to the computation algorithm.
There are various drawbacks to these prior art approaches. These approaches cannot handle incremental changes to the data in database 100 and database 200 efficiently. Both approaches require a full reload of the entire data set to accommodate situations where even only a small portion of the data in database 100 or database 200 has been changed. It has heretofore not been possible to build a computationally intensive application that simultaneously supports high frequency interactions, from a user or otherwise, and access to the most current changes to information.